Media Flow Experience using Influential Feature Analysis

Introduction Understanding and modeling human experience and emotional response is important for bridging the gap between the “shallow” ways of encoding or interpreting information by a machine and the rich meaning that humans are able to capture when they listen or view audiovisual content. The ability to analyze media data in terms emotional or affective attributes opens the door to a whole new realm of intelligent applications, ranging from query-by-emotion for audio retrieval, affective interaction with computers, such as gaming or improvisation, and suggesting new methods for active or generative media content creation the might operate by dynamically changing its contents according to affective user responses. One of the fundamental problems in formalizing our intuitive ideas about emotions and affect is related to apparently strictly technical notions of quantifying information contents embedded in a message. It become increasingly evident that information does not have an objective or universal definition. Already more then fifty year ago, in an introduction to Shannon’s classic paper [1], Warren Weaver claimed that the mathematical theory of communication does not touch the entire realm of information processing, which he stratified that exists on three levels: technical, semantic, and influential. The idea behind influential aspects of information is that the meaning of signals is determined from actions, such as classification or estimation taken as a result of interpreting the information [2][3]. The emotional or affective content of a message might be considered influential in the sense that they operate to alter the state of the system that processes information. In order for a system to be influenced, it must be capable of altering its operational state, which in turn may be achieved by self-monitoring of the successes or failures in its comprehension of the incoming messages. In this paper we develop one such framework, trying to relate several higher signal features to psycho-acoustic results of human experience when listening to music. The idea leads to a general formulation of “media flow experience”, a new notion that allows describing the evolution and unfolding of musical material in time in terms of mental and affective attributes.